Imaging apparatuses called light field cameras and the like are drawing attention these days. Such an imaging apparatus obtains captured images of viewpoints through imaging units, generates a captured image of a virtual viewpoint from the captured images (view interpolation), and generates a captured image of a virtual focus or the like using these captured images (see Patent Document 1, for example).
Such an imaging apparatus generates an image of a virtual viewpoint by generating the parallax images (depth maps) corresponding to captured images of viewpoints, generating a captured image of the virtual viewpoint from the captured images using the parallax images for the respective viewpoints (image registration), and combining the captured images. Therefore, to generate a high-precision captured image of the virtual viewpoint, it is necessary to generate the parallax images with high precision. It should be noted that a parallax image is an image having pixel values that are the disparity values of the respective pixels in a captured image. Here, an image of an object located on the farther side has a smaller disparity value, and an image of an object located on the nearer side has a larger disparity value.